Apparatus and method for managing user inputs in video games

ABSTRACT

A system that incorporates teachings of the present disclosure may include, for example, a computing device having a controller to obtain a user input that was inputted into a first accessory operably coupled with the computing device where the first accessory provides a user interface for user interaction with a video game, determine a language of an intended recipient of the user input based on an identity of the intended recipient, access a multi-lingual library comprising a plurality of words associated with the video game, match the user input to one or more words of the plurality of words of the multi-lingual library to generate a translated message in the determined language of the intended recipient, and provide the translated message to a second accessory for presentation to the intended recipient in real-time. Additional embodiments are disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/924,127 filed Oct. 27, 2015, which is acontinuation of and claims priority to U.S. patent application Ser. No.14/573,539 filed Dec. 17, 2014 (now U.S. Pat. No. 9,199,174), which is acontinuation of and claims priority to U.S. patent application Ser. No.13/683,737, filed Nov. 21, 2012 (now U.S. Pat. No. 8,939,837), which isa continuation of and claims priority to U.S. patent application Ser.No. 12/947,874 filed Nov. 17, 2010 (now U.S. Pat. No. 8,337,305), thedisclosures of all of which are incorporated herein by reference intheir entirety.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to management applications, andmore specifically to an apparatus and method for managing user inputs invideo games.

BACKGROUND

It is common today for video games to allow multiple gamers to playsimultaneously, such as online or offline. This is especially true ofgamers who play Massively Multiplayer On-line (MMO) games in a team orindividual configuration.

These gamers can be located in different areas, including differentcountries, and may or may not know each other.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an illustrative embodiment of a communication systemaccording to the present disclosure;

FIG. 2 depicts an illustrative embodiment operating in portions of thesystem of FIG. 1;

FIGS. 3-5 depict illustrative embodiments of a Graphical User Interface(GUI) generated by an Accessory Management Software (AMS) applicationaccording to the present disclosure;

FIGS. 6-8 depict illustrative methods describing the operation of theAMS application; and

FIG. 9 depicts an illustrative diagrammatic representation of a machinein the form of a computer system within which a set of instructions,when executed, may cause the machine to perform any one or more of themethodologies disclosed herein.

DETAILED DESCRIPTION

The embodiments described herein illustrate devices and/or methodologiesthat allow gamers to communicate with each other, even when the gamersspeak different languages. One or more of the embodiments can translateuser inputs into other languages that are identified with the intendedrecipient(s). The user inputs can be speech and/or non-speech (e.g.,actuations of accessory buttons such as depressing a combination of keysthat represent particular words). The translation can be performed by anumber of different devices, including the source device, the recipientdevice, and/or a remote server (e.g., a service provider server that isin communication with the source and recipient devices, such as via theInternet). One or more of the embodiments allow for configuration of aprocessor with the identity of gamers and their respective languages.The processor can also be configured to detect certain words or phrases,which can be translated to the language of each other gamer. One or moreof the embodiments can provide for transmission of codes to otherprocessors, which then locally invokes translation. Other embodimentsare also contemplated.

One embodiment of the present disclosure entails a non-transitorycomputer-readable storage medium that includes computer instructions toreceive a signal generated in response to a user input at a firstaccessory operably coupled with a first computing device, where thefirst computing device is programmed to present a video game. Thecomputer instructions can also determine a language for an intendedrecipient of the user input and can provide an audio signal to a secondaccessory for presentation to the intended recipient. The audio signalcan be generated based on the received signal and the determinedlanguage using a multi-lingual library. The audio signal can beassociated with the video game.

One embodiment of the present disclosure entails a method includingobtaining a user input that was inputted into a first accessory operablycoupled with a first computing device, where the first computing deviceis programmed to present a video game. The method can also includedetermining a language of an intended recipient of the user input basedon an identity of the intended recipient and translating the user inputinto a translated message using the determined language and amulti-lingual library. The method further can include providing thetranslated message to a second accessory for presentation to theintended recipient in real-time, where the translated message isassociated with the video game.

One embodiment of the present disclosure entails a computing devicehaving a controller to obtain a user input that was inputted into afirst accessory operably coupled with the computing device, where thefirst accessory provides a user interface for user interaction with avideo game. The controller can also be programmed to determine alanguage of an intended recipient of the user input based on an identityof the intended recipient and access a multi-lingual library comprisinga plurality of words associated with the video game. The controller canfurther be programmed to match the user input to one or more words ofthe plurality of words of the multi-lingual library to generate atranslated message in the determined language of the intended recipientand provide the translated message to a second accessory forpresentation to the intended recipient in real-time.

FIG. 1 depicts an illustrative embodiment of a communication system 100for delivering media content, which can include video games. It iscontemplated by the present disclosure that other media content can bedelivered by the system 100, such as, for example, audio content, movingimage content (e.g., 2D or 3D videos), still image content, andcombinations thereof. It is contemplated that the present disclosure canapply to any present or next generation over-the-air and/or landlinemedia content services system. Multiple forms of media services can beoffered to media devices over landline technologies such as thosedescribed above. Additionally, media services can be offered to mediadevices by way of a wireless access base station operating according towireless access protocols, such as Wireless Fidelity (WiFi), or cellularcommunication technologies such as Global System for Mobile or GSM, CodeDivision Multiple Access or CDMA, Time Division Multiple Access or TDMA,Universal Mobile Telecommunications or UMTS, World interoperability forMicrowave or WiMAX, Software Defined Radio or SDR, Long Term Evolutionor LTE, and so on). Other present and next generation wide area wirelessnetwork technologies are contemplated by the present disclosure.

System 100 can include a first gaming device 120 having one or moreaccessories (e.g., first accessory 125 and second accessory 135)operably coupled thereto. The first gaming device 120 can be of varioustypes of computing devices, including a gaming console, a desktopcomputer, a laptop computer, a server, a mainframe computer, a cellularphone, a personal digital assistant, and/or a media player. Theaccessories 125 and 135 can be provided with components for the deliveryof messages between gamers. For instance, first and second accessories125 and 135 can each include a headset 140 with a microphone andspeaker(s), which allow for input of a voice message and playing ofvoice messages. The headsets 140 can be wired and/or wireless devices.It should be understood that the headsets 140 are one example ofcomponents that can be used for delivery of audio messages, but thepresent disclosure contemplates that other components can also beutilized, such as a microphone and speaker integrated into the firstaccessory and/or the gaming console.

System 100 can also include one or more second gaming devices 130 incommunication with the first gaming device 120. Each of the secondgaming devices 130 can include the second accessories 135 with headsets140 or other components to facilitate receipt and delivery of messagesbetween gamers. The physical location of the gaming devices 120 and 130can vary. For instance, the gaming devices 120 and 130 can be remotefrom each other and/or can be located within the same area or building.

The gaming devices 120 and 130 can be in communication with each othervia network 150. The type of network 150 can vary. For example, network150 can include the Internet and/or a service provider network. Inanother embodiment, network 150 can include a home or Local AreaNetwork. In another embodiment, the network 150 can be a hardwireconnection between gaming devices located in proximity to each other.Network 150 can comprise various devices (such as routers, gateways,servers and so forth). Network 150 can operate according to variousprotocols (such as Ethernet, IP, TCP, UDP, ICMP, HTTP, FTP, IMAP, and soforth) in order to allow delivery of messages between the gaming devices120 and 130 and/or establishing gaming sessions between the devices suchthat users of the first and second gaming devices can interact in thevideo game.

In one embodiment, first gaming device 120 can include, or otherwise bein communication with, a translator 190 for translating messages from auser of the first accessory 125. The messages can be voicecommunications that are received by the first gaming device 120, such asthrough headset 140. The messages can also be generated withoutrecording, or otherwise receiving, a voice communication from the userof first accessory 140. For instance, a user input at first accessory125 can be non-speech, such as depressing particular a button(s) of thefirst accessory that generates a message. In one embodiment, macros canbe configured which represent messages to be delivered to other gamers.The macros can then be implemented by the user of first accessory 125 bya particular manipulation of the first accessory, such as depressing aparticular combination of buttons.

Translator 190 can be integrated with the first gaming device 120, suchas a controller of the gaming device being programmed to function as thetranslator 190, and/or can be a separate device. In one embodiment, thetranslator 190 can be a separate device that is removably connectablewith the first gaming device 120, such as a removable adaptor that canbe plugged into a port of the gaming device, such as a USB port. Inanother embodiment, the translator 190 can be in wireless communicationwith the first gaming device 120. In yet another embodiment, thetranslator 190 can be directly in communication with the first accessory125 or can be in direct communication with the user of the firstaccessory 125 for receiving the messages.

Translator 190 can translate the message into one or more languages forpresentation to the other gamers. For example, where the user input is avoice message then the translator 190 can apply voice recognition todetermine the words of the message. As another example, where themessage is a macro of an accessory or is a text message (such as enteredusing a keyboard or other text interface) then the translator need notapply voice recognition.

Translator 190 can identify corresponding words in another language fordelivery to the intended recipients of the voice message. In oneembodiment, the translator 190 can utilize a multi-lingual library,which includes a plurality of words in various languages. Themulti-lingual library can be utilized as a look-up table for translatingthe message between languages. The library can be stored at variouslocations. For example, the library can be stored with the translator190 and/or with a memory of the first gaming device 120. In anotherembodiment, the library can be stored remotely from the translator 190and accessed when a translation is being performed. For example, acentralized library can be maintained using a remote server that isaccessible by the translator 190, such as via the network 150. Inanother embodiment, the library can be a distributed database that ismaintained by a number of different servers and accessible by translator190.

Translator 190 can also apply other translation techniques intranslating the message between languages. For example, in oneembodiment the translator can apply grammatical rules to the messagetranslation to achieve proper word order and so forth. In anotherembodiment, the translator 190 can correct messages for improper grammaror terminology, such as improper grammar in the original message ormisspelled words in a text message.

In one embodiment, translator 190 of the first gaming device 120 can beutilized to translate messages being transmitted between gamers usingthe same first gaming device 120. In this example, the translation anddelivery of the messages may be performed without utilizing the network150, such as the first gaming device 120 directly receiving the originalmessage from the first accessory 125 and/or the headset 140 and directlydelivering the translated message to the headset 140 of the secondaccessory 135. However, the present disclosure contemplates utilizingthe network 150 for translation between gamers of the same gamingdevice, such as to access the multi-lingual library or to access aremote server which performs the translation.

In another embodiment, the translator 190 of first gaming device 120 canreceive the user input of the message and can translate the messageusing the multi-lingual library for delivery to the second gamingdevices 130. The delivery of the translated messages can be by way ofthe network 150. In one embodiment, codes representative of a translatedaudio message can be transmitted to the second gaming devices 130 ratherthan transmitting the translated audio message. The codes can then beutilized by the second gaming devices to present the translated message,such as in a synthesized voice or text.

In yet another embodiment, the second gaming devices 130 can receive theuser input and can perform the translation of the message utilizing atranslator 190 of the second gaming devices. For instance, where theuser input is a voice message recorded by headset 140 of first accessory125, the second gaming devices 130 can receive audio signals of thevoice message via the network 150. The translator 190 of each of thesecond gaming devices 130 can then apply voice recognition to the audiosignals to generate a translated message for presentation to the gamersof the second gaming devices. As another example, where the user inputis a text message or a message generated by a macro of the firstaccessory 125, the second gaming devices 130 can receive data signalsrepresentative of the message. Translator 190 can then translate theuser input into the translated message without using voice recognition.

In another embodiment, system 100 can utilize a translator 190 of aremote server 175 for performing translation of messages between thegamers of the gaming device s 120 and 130. Similar to the messageformats describes above, translator 190 of the remote server 175 canapply voice recognition to audio messages received from the first gamingdevice 120 or can perform translation without using voice recognitionwhere a text message or accessory macro is received from the firstgaming device. In one embodiment, the remote server 175 can transmit anaudio and/or text message to the second gaming devices 130. In anotherembodiment, the remote server 175 can transmit code signals thatrepresent the translated message, where the second gaming devicesgenerate the translated message based on the code signals.

The present disclosure contemplates the use of various devices andcombinations of devices to implement translator 190 and delivertranslated messages between gamers. In one embodiment, network or deviceperformance and/or other factors can be utilized to determine whichtranslators 190 are to perform the translation. As an example, a gamingsession can include a first gamer using the first gaming device 120, asecond gamer using the second gaming device 130, and a third gamer usinganother second gaming device 130. The first gamer can transmit a messageto the second gamer which is translated into the second gamer'slanguage. The translation into the second gamer's language can beperformed by a translator 190 of the first gaming device 120. However,to conserve computing resources of the first gaming device 120, themessage can be translated into a different language of the third gamerby the other second gaming device 130 of the third gamer. In thisexample, the first gaming device 120 does not expend computing resourcestranslating a message into the third gamer's language since the thirdgamer was not the intended recipient of the message. However, the thirdgamer can still participate in the communication, but does so throughutilizing his or her own computing resources.

FIG. 2 depicts an illustrative method 200 operating in portions of thesystem 100 of FIG. 1. Method 200 can begin with step 202 in which a userinput of a message is obtained. The user input can be of variousformats, including a voice message, a text message and/or a macrogenerated by an accessory device where the macro represents a message.The message can be associated with a video game. For example, messagesrelated to game play can be communicated between gamers using method200.

In step 204, a language for the one or more intended recipients of themessage can be determined. For example, languages can be designated ingamer profiles that are accessible by each of the gaming devices. Othertechniques can be utilized to determine the language of the gamer,including transmitting requests between gaming devices. In anotherembodiment, voice communications transmitted from a particular gamer canbe utilized to determine the language of the particular gamer.

In step 206, the translator 190 can access a multi-lingual library toperform the translation of the user input. As described above, theparticular device implementing the translator 190 can vary and caninclude the source device, the recipient device and/or a remote server.Where the user input is a voice message then the translator 190 canapply voice recognition to determine words of the untranslated message.If the user input is not speech then the translator 190 can directlyutilize the message along with the multi-lingual library.

In one embodiment in step 208, the translator 190 can utilize a contextof the video game to assist in translation. For instance, a context ofthe video game can be determined based on the graphics being shown atthe time the message was input by the user. As an example, a user may bemoving along a beach as part of a number of different environments in aparticular video game. If the user transmits a voice message stating“swim”, the translator 190 can determine the context (e.g., beach withwater) and can apply voice recognition in combination with the contextto distinguish between “swim” and other words, such as “win.” Thecontext of the video game can also be applied to facilitate thetranslation other than when using voice recognition. For example, a userinput may be a text message that is a misspelling, such as “wim.” Basedon the context of the video game at the time of the message being abeach with water, the translator can again distinguish between “swim”and “win.”

If in step 210 the translator 190 identifies a match for the message inthe multi-lingual library then method 200 proceeds to step 212 where thetranslation of the message is presented to the intended recipient. Thepresentation of the translated message can be performed in a number ofdifferent ways. For example, the translated message can be a humanvoice, synthesized speech and/or text (e.g., presented on the displaydevice with the video game). In one embodiment, the format of thepresentation can be selected by one of the source and the recipient ofthe message. In another embodiment, the translator 190 can apply grammarrules to matching words of the multi-lingual library to facilitate thetranslation.

If on the other hand in step 210 the translator 190 cannot identify amatch for the message in the multi-lingual library then method 200 canproceed to step 214 where the untranslated message is presented to theintended recipient. In another embodiment, if a match is not determinedthen the translator 190 can request a search of other multi-linguallibraries. For example, if the first gaming device 190 cannot find amatch for a voice message generated by a first gamer then the firstgaming device can access or otherwise request a search of themulti-lingual library of the second gaming device 130 and/or the remoteserver 175 via the network 150.

Method 200 can present multiple translations of the same message tomultiple gamers. The multiple translations can be performed by the sametranslator 190 or by different translators, such as using a distributedtranslation technique.

In one embodiment, the multi-lingual library can be populated based onwords or phrases that are commonly used with particular video games. Thelibrary can be updated as new words or phrases are being used duringgaming sessions. For example, where software updates including newactions or new graphics are provided to a video game, new words orphrases may more commonly be used with the updated video game. Thelibrary can be updated with translations for these new words or phrases.

In another embodiment, the multi-lingual library can be populated basedon words or phrases suggested by the user. For example, a user canrequest particular words or phrases be translated into particularlanguages. The translation of the words or phrases can be determined andthe library can be provisioned with these translations. In anotherembodiment, an absence of matches for a particular word or phrasemeeting a threshold can trigger populating the library with atranslation of the word or phrase.

FIGS. 3-5 depict illustrative embodiments of a Graphical User Interface(GUI) generated by an Accessory Management Software (AMS) applicationaccording to the present disclosure. The AMS application can operate ina computing device such as a desktop computer, a laptop computer, aserver, a mainframe computer, and/or a gaming console. A gaming consolecan represent a gaming device such as a Playstation 3™, a Wii™, or anXbox360™. Other present and next generation gaming consoles arecontemplated. The AMS application can also operate in other computingdevices with less computing resources such as a cellular phone, apersonal digital assistant, or a media player (such as an iPOD™). Fromthese illustrations it would be apparent to an artisan with ordinaryskill in the art that the AMS application can operate in any device withcomputing resources.

FIGS. 6-8 depict illustrative methods 600-800 describing the operationof the AMS application as shown in FIGS. 3-5. Method 600 can begin withstep 602 in which the AMS application is invoked in a computing device.The invocation step can result from a user selection of the AMSapplication from a menu or iconic symbol presented on a desktop of thecomputing device by an operating system (OS) managing operationsthereof. In step 604, the AMS application can detect by way of driversin the OS a plurality of operationally distinct accessoriescommunicatively coupled to the computing device. The accessories can becoupled to the computing device by a tethered interface (e.g., USBcable), a wireless interface (e.g., Bluetooth or WirelessFidelity—WiFi), or combinations thereof.

In the present context, an accessory can represent any type of devicewhich can be communicatively coupled to the computing device and whichcan control aspects of the OS and/or a software application operating inthe computing device. An accessory can represent for example a keyboard,a gaming pad, a mouse, a gaming console controller, a joystick, amicrophone, or a headset with a microphone—just to mention a few. Thekeyboard and gaming pad represent accessories of a similar categorysince their operational parameters are alike.

A mouse, on the other hand, represents an accessory having disparateoperational parameters from the keyboard or gaming pad. For instance,the operational parameters of a keyboard generally consist ofalphanumeric keys, control keys (e.g., Shift, Alt, Ctrl), and functionkeys while the operational parameters of a mouse consist of navigationdata generated by a tracking device such as a laser sensor, buttons toinvoke GUI selections, and settings thereof (e.g., counts or dots perinch, acceleration, scroll speed, jitter control, line straighteningcontrol, and so on). Such distinctions can be used to identify disparatecategories of accessories. The joysticks, game controllers or any otherinput devices represent additional categories of accessories supportedby the AMS.

In step 606, the AMS application presents a GUI 301 such as depicted inFIG. 3 with operationally distinct accessories such as the keyboard 308and mouse 310. The GUI 301 presents the accessories 308-316 in ascrollable section 317. Although the present disclosure contemplates theuse of other GUI configurations. One or more accessories can be selectedby a user with a common mouse pointer. In this illustration, thekeyboard 308 and mouse 310 were selected with a mouse pointer forcustomization. Upon selecting the keyboard 308 and mouse 310 in section317, the AMS application presents the keyboard 308 and mouse 310 insplit windows 318, 320, respectively, to help the user during thecustomization process.

In step 608, the AMS application can be programmed to detect auser-selection of a particular software application such as a game. Thisstep can be the result of the user entering in a Quick Search field 360the name of a gaming application (e.g., World of Warcraft™). Uponidentifying a gaming application, the AMS application can retrieve instep 610 from a remote or local database gaming application actionswhich can be presented in a scrollable section 339 of the GUIrepresented as “Actions” 330. The actions can be tactical actions 332,communication actions 334, menu actions 336, and movement actions 338,or any other types of actions, which can be used to invoke and managefeatures of the gaming application.

The actions presented descriptively in section 330 of the GUI canrepresent a sequence of accessory input functions which a user canstimulate by button depressions, navigation or speech. For example,depressing the left button on the mouse 310 can represent the tacticalaction “Reload”, while the simultaneous keyboard depressions “Ctrl A”can represent the tactical action “Melee Attack”. For ease of use, the“Actions” 330 section of the GUI is presented descriptively rather thanby a description of the input function(s) of a particular accessory.

Any one of the Actions 330 can be associated with one or more inputfunctions of the accessories by way of a simple drag and drop action.For instance, a user can select a “Melee Attack” by placing a mousepointer 333 over an iconic symbol associated with this action. Upondoing so, the symbol can be highlighted to indicate to the user that theicon is selectable. At this point, the user can select the icon byholding the left mouse button and drag the symbol to any of the inputfunctions (e.g., buttons) of the keyboard 308 or mouse 310 to make anassociation with an input function of one of these accessories.

For example, the user can drag the Melee Attack symbol to the rightmouse button thereby causing an association between the selection of theright mouse button and the gaming action of a Melee Attack. When theright button of the mouse 310 is selected during normal operation, theAMS application can detect the selection as a “trigger” to generate thekey sequence “Ctrl A” which is understood by the gaming application asrequest for a Melee Attack. The gaming application receives from the AMSapplication by way of an operating system the “Ctrl A” sequence as if ithad been generated by a Qwerty keyboard.

With this in mind, attention is directed to step 612 where the AMSapplication can respond to a user selection of a profile. A profile canbe a device profile or master profile invoked by selecting GUI button356 or 358, each of which can identify the association of actions withinput functions of one or more accessories. If a profile selection isdetected in step 612, the AMS application can retrieve macro(s) and/orprior associations of actions with the accessories as defined by theprofile. The actions and/or macros defined in the profile can also bepresented in step 616 by the AMS application in the actions column 330of the GUI 301 to modify or create new associations.

In step 618, the AMS application can also respond to a user selection tocreate a macro. A macro in the present context can represent a subset ofactions that can be presented in the Actions column 330. Any commandwhich can be recorded by the AMS application can be used to define amacro. A command can represent a sequence of input functions of anaccessory, identification of a software application to be initiated byan operating system (OS), or any other recordable stimulus to initiate,control or manipulate software applications. For instance, a macro canrepresent a user entering the identity of a software application (e.g.,instant messaging tool) to be initiated by an OS.

A macro can also represent a message to be delivered to another gamer.GUI 301 can be used to configure delivery of the message. For instance,the message can be recordable speech delivered by a microphone to one ormore other gamers. As another example, the message can be synthesizedspeech delivered to one or more other gamers. In yet another example,the message can be text delivered. GUI 301 allows particular actuationsof the accessory to represent the particular message. For instance, themessage “duck” can be programmed as a macro using the GUI 301. Themessage can then be delivered to the other gamers when the sourceaccessory is actuated in such a way to request delivery of the message(e.g., depressing the corresponding button(s) associated with theuser-generated macro). Other preferences associated with the message canalso be selected using GUI 301. For instance, a type of voice (e.g.,female or male voice) can be selected for messages that are delivered asaudio signals which are not in the voice of the user. As another exampleof preference selection, a font can be selected by a user for messagesthat are delivered as text.

In another embodiment, the macro and corresponding message can begenerated and stored by the gaming console without user provisioning,such as based on messages that have been already associated with aparticular video game. For instance, a video game can be associated witha set of typical user messages, such as “duck”, “run”, “stay here” andso forth. These messages can further be associated with particularaccessory actuations, by the user and/or pre-configured. In one example,GUI 301 can present a list of typical messages for a video game and auser can select the accessory actuation to deliver the message. Inanother example, the accessory actuation can be pre-configured by thesystem with the messages. In one embodiment, GUI 301 can be used tochange the pre-configuration of the message macros, including changingthe corresponding accessory actuations or removing one or more messagesfrom the group of messages that can be delivered.

As described above with respect to system 100, the message can betranslated to facilitate communication with other gamers speaking one ormore different languages. The translation can be performed by the sourcedevice, the recipient device, and/or a remote server in communicationwith the source and recipient devices. In one embodiment, GUI 301 can beused to configure translations of the messages. For instance, a user maydesire only certain translations of the user's messages be performed,including based on a selection of words that are to be translated or aselection of languages that are to be utilized for the translation. Inone embodiment, GUI 301 can also be used to designate particular wordsthat are not to be translated.

As another example, GUI 301 can be used so that a preferred language isdesignated by a user. In one embodiment, source devices that intend tosend a message to the other gamers can then utilize the preferredlanguage information (which can be delivered to the other gamer devicessuch as when the gaming session is established that includes the variousparties) to perform the translation. In another embodiment, the GUI 301can provide the preferred language information to a remote server, whichperforms translations of messages associated with a gaming session.

In yet another embodiment a macro can represent recordable navigation ofan accessory such as a mouse or joystick, recordable selections ofbuttons on a keyboard, a mouse, or a mouse pad, and so on. Macros canalso be combinations of the above illustrations. Macros can be createdfrom the GUI 301 by selecting a “Record Macro” button 348. The macro canbe given a name and category in user-defined fields 340 and 342.

Upon selecting the Record Macro button 348, a macro can be generated byselection of input functions on an accessory (e.g., Ctrl A, speech,etc.) and/or by manual entry in field 344 (e.g., typing the name andlocation of a software application to be initiated by an OS). Once themacro is created, it can be tested by selecting button 350 which canrepeat the sequence specified in field 344. The clone button 352 can beselected to replicate the macro sequence if desired. Fields 352 can alsopresent timing characteristics of the stimulation sequence in the macrowith the ability to customize such timing. Once the macro has been fullydefined, selection of button 354 records the macro in step 620. Therecording step can be combined with a step for adding the macro to theassociable items Actions column 330, thereby providing the user themeans to associate the macro with input functions of the accessories.

In step 622, the AMS application can respond to drag and dropassociations between actions and input functions of the keyboard 308 andthe mouse 310. If an association is detected, the AMS application canproceed to step 624 where it can determine if a profile has beenidentified in step 612 to record the association(s) detected. If aprofile has been identified, the associations are recorded in saidprofile in step 626. If a profile was not been identified in step 612,the AMS application can create a profile in step 628 for recording thedetected associations. In the same step, the user can name the newlycreated profile as desired. The newly created profile can also beassociated with one or more software applications in step 630 for futurereference.

The GUI 301 presented by the AMS application can have other functions.For example, the GUI 301 can provide options for layout of the accessoryselected (button 322), how the keyboard is illuminated when associationsbetween input functions and actions are made (button 334), andconfiguration options for the accessory (button 326). Configurationoptions can include operational settings of the mouse 310 such as DotsPer Inch or Counts Per Inch, and so on. The AMS application can adaptthe GUI 301 to present more than one functional perspective. Forinstance, by selecting button 302, the AMS application can adapt the GUI101 to present a means to create macros and associate actions toaccessory input functions as depicted in FIG. 3. Selecting button 304can cause the AMS application to adapt the GUI 301 to present statisticsin relation to the usage of accessories as depicted in FIGS. 4-5.Selecting button 306 can cause the AMS application to adapt the GUI 301to present promotional offers and software updates.

It should be noted that the steps of method 600 in whole or in part canbe repeated until a desirable pattern of associations of actions toinput functions of the selected accessories has been accomplished. Itwould be apparent to an artisan with ordinary skill in the art thatthere can be numerous other approaches to accomplish similar results.These undisclosed approaches are contemplated by the present disclosure.

FIG. 7 depicts a method 700 in which the AMS application can beprogrammed to recognize unknown accessories so that method 600 can beapplied to them as well. Method 700 can begin with step 702 in which theAMS application detects an unknown accessory such as a new keyboard froman unknown vendor by way of a communicative coupling to a computingdevice from which the AMS application operates. The AMS application inthis instance can receive an identity from the keyboard or the operatingsystem which is not known the AMS application. Upon detecting an unknownaccessory, the AMS application in step 704 can present a depiction of anaccessory of similar or same category in response to a user providingdirection as to the type of accessory (by selecting for example adrop-down menu). Alternatively, or in combination with the userinstructions, the AMS application can determine from the informationreceived from the unknown accessory an accessory type.

In step 706 the AMS application can receive instructions describing allor a portion of the input functions of the unknown accessory. Theseinstructions can come from a user who defines each input functionindividually or responds to inquiries provided by the AMS application.The AMS application can for example make an assumption as to thekeyboard layout and highlight each key with a proposed function whichthe user can verify or modify. Once the AMS application has beenprovided instructions in step 706, the AMS application can create anaccessory identity in step 708 which can be defined by the user. Insteps 710 and 712, the AMS application can associate and record theaccessory instructions with the identity for future recognition of theaccessory. In step 714, the AMS application can present a depiction ofthe new accessory with its identity along with the other selectableaccessories in section 317.

Method 700 can provide a means for universal detection andidentification of any accessory which can be used to control or managesoftware applications operating in a computing device.

FIG. 8 depicts a method 800 for illustrating the AMS applicationresponding to input function stimuli (triggers) of accessories. Method800 can begin with step 802 in which the AMS application monitors theuse of accessories. This step can represent monitoring the stimulationof input functions of one or more accessories communicatively coupled toa computing device from which the AMS application operates. The inputfunctions can correspond to button depressions on a keyboard, gamingpad, or navigation device such as a mouse. The input functions can alsorepresent navigation instructions such as mouse or joystick movements.The input functions can further represent speech supplied by amicrophone singly or in combination with a headset. Other existing orfuture input functions of an accessory detectable by the AMS applicationare contemplated by the present disclosure. The AMS application canmonitor input functions by for example processing human interface device(HID) reports supplied by the accessories to the computing device.

Once one or more stimulations have been detected in step 804, the AMSapplication can proceed to step 806 to determine if action(s) have beenassociated with the detected stimulation(s). If for example thestimulations detected correspond to keyboard and mouse buttondepressions, the AMS application can determine if actions have beenassociated and recorded for such stimulations. If these stimulations“trigger” one or more actions, the AMS application can proceed to step808 where it retrieves the stimulation definition of these actions foreach accessory reporting a stimulation. In step 810, the AMS applicationcan substitute the detected stimulations with the stimulations definedby the action.

To illustrate this substitution, suppose for example that the detectedstimulation was “Ctrl A” simultaneously depressed on a keyboard. Supposefurther that an action associated with this stimulus consists of a macrothat combines mouse clicks with a navigation of the mouse (e.g., movingthe mouse quickly in a forward motion for a given distance), and arequest to invoke an instant messaging (IM) session with a particularindividual using Skype™ or some other common IM tool. In step 810, theAMS application would substitute “Ctrl A” for stimulations consisting ofthe mouse clicks, navigation and a request for an IM application. Thesubstitute stimulations would then be reported in step 812 to anoperating system (OS).

In step 816, the OS can determine whether to pass the substitutestimulations to an active software application in operation (e.g., agaming application) and/or to invoke another software application. Theactive software application can be operating from the same computersystem from which the OS and the AMS application operate or can beoperating at a remote system such as an on-line server or family ofservers (e.g., World of Warcraft) awaiting stimulation data from thecomputer system. In this illustration, the macro comprises bothstimulation feedback for the active software application and a requestto initiate an IM session. Accordingly, the OS conveys in step 818 themouse stimulation signals to the active software application (e.g.,gaming application), and in a near simultaneous fashion invokes the IMsession in step 820 with a specific individual (or organization).

Referring back to step 806, the illustrations above cover a scenario inwhich the AMS application has detected an association of actions toaccessory stimuli. If however the AMS application does not detect suchan association, then the detected stimulus (or stimuli) supplied by oneor more accessories is transmitted to the OS in step 814. For example,it may be that a stimulation based on the depressions of “Ctrl A” has noparticular association to an action. In this case, the AMS applicationpasses this stimulation to the OS with no substitutes. In step 816 theOS can determine if this stimulation invokes a new software applicationin step 820 or is conveyed to the previously initiated softwareapplication.

Contemporaneous to the embodiments described above, the AMS applicationcan also record in step 822 statistics relating to the detectedaccessory stimulations. A portion of the AMS application can operate asa background process which performs statistical analysis on thestimulations detected. By selecting button 304 in FIG. 3, the AMSapplication can provide an updated GUI which illustrates the usage ofinput functions of one or more accessories for which stimulations weredetected in step 804. For ease of illustration, only a keyboardaccessory is shown. In this illustration, certain keys (references 404,406, 408, 410) on the keyboard are color-coded to illustrate thefrequency of usage of these keys.

A color scale 403 defines the frequency of usage of the input functionsof the keyboard. The first end of the scale (navy blue) represents asingle detected depression, while an opposite end of the scale (brightred) represents detected depressions. Based on this scale, the AMSapplication maps by color in step 824 stimulations of the keyboard. Forexample, the key grouping 408 depict a color coding with the highestdetectable usage, while the F7 key (reference 410) indicates the fewestdepressions. Keys having zero depressions are not color coded to readilyidentify the color mapping of keys which were used at least once.

The AMS application provides additional functions in a playback panel ofthe GUI which can help a user understand how the color coded keys wereused during an active software application such as a video game. In thissection of the GUI, the AMS application can present the user with aplayback control function 402 which the user can select to replay,pause, forward or rewind the usage of these keys. When usage playback isselected, the user can for instance see the color coded keys highlightedin real-time with a temporary white border to visualize how the keyswere selected. A time clock 404 provides the user the elapsed time ofthe playback sequence. Button 412 allows the user to retrieve statisticsfrom other sessions, while button 414 provides the user a means to savestatistics from a given session.

The GUI of FIG. 4 could have been shown as a split screen with allaccessories which generated one or more detected stimulations (e.g.,keyboard, mouse, and microphone), each providing statistical symbolicresults as described above for the keyboard. Although not shown, splitscreen embodiments are contemplated by the present disclosure for theGUI of FIG. 4.

In addition to a symbolic representation as shown in FIG. 4, the AMSapplication can provide the user a means to visualize raw statistics ina table format such as shown in FIG. 5 by selecting button 412. Thetable format shows raw data in section 502 and possible suggestions insection 504 for improving user performance which can be generated by theAMS application in step 826. Section 502 can be presented in a tableformat with a column identifying the key being analyzed, its usage, andnumber of key presses. The user can ascertain from this table the mostand least frequently used keys as well as other identifiable patterns.

The AMS application can utilize an understanding of the layout of theaccessory (in this case, the keyboard) to determine from the statisticsways that the user can improve response time or ergonomic use. Forexample, the AMS application can determine from a layout analysis thatthe key combination <Alt.> can be reassigned to a macro based on thetrigger <Ctrl F> which could provide the user a faster response time andfree up the user's right hand for other tasks. The AMS application canalso provide alternative suggestions. For example, the AMS applicationcan also suggest creating single button macros for each of the keycombinations <Alt.> and <Ctrl A> which can be assigned to keys on thekeyboard or left and right buttons of a mouse. The latter suggestion ofassigning macros to the mouse can help the user free up his/her lefthand.

The AMS application can utilize present and next generation algorithmsto determine how to improve response times and ergonomic usage ofaccessory devices. The AMS application can for example have at itsdisposal an understanding of the layout of each accessory, the type ofsoftware being controlled by the accessory (e.g., World of Warcraft),type of operations commonly used to control the software (e.g., knownactions as shown in the actions column 330 of FIG. 3), an understandingof the associations made by other users (e.g., gamers) to improve theirperformance when controlling the software, and so on. The AMSapplication can also be adapted to communicate with the active softwareapplication by way of an Application Programming Interface (API) toreceive additional usage statistics from the software which it can inturn use to improve the user's performance. The AMS application can alsoutilize common statistical and behavior modeling techniques to predictthe behavior of the user and responses from the software application toidentify possible ways to improve the user's performance.

From these illustrations, it would be apparent to an artisan of ordinaryskill in the art that innumerable algorithms can be developed to analyzeaccessory usage and thereby suggest improvements. These undisclosedembodiments are contemplated by the present disclosure.

From the foregoing descriptions, it would be evident to an artisan withordinary skill in the art that the aforementioned embodiments can bemodified, reduced, or enhanced without departing from the scope andspirit of the claims described below. For example, method 600 can beadapted to define more than one programmable layer for an accessory.Such a feature can extend the functionality of an accessory intomulti-layer paradigms of input functions. The GUI of FIG. 3 can beadapted so that a user can specify more than one programmable layer fora specific accessory.

The user can also specify which layer to present in FIG. 3 whileassociating actions. If for instance layer 1 is shown, the GUI of FIG. 3can present the actions associated in this layer by presentingdescriptors superimposed on the input functions (e.g., buttons or keys).When the user switches to layer 2 (e.g., by selecting from a drop-downmenu the layer of interest) the accessory can be shown in the GUI with adifferent set of associated actions. The user can define a macro oridentify a key sequence to switch between layers when the accessory isin use.

The trigger for switching between layers can be a toggle function (e.g.,selecting the tab key on a Qwerty keyboard) to switch between layers ina round robin fashion (layer 1→layer 2→layer 3→ to layer 1→ and so on).Alternatively, the user can define a hold and release trigger to switchbetween layers. In this embodiment, the user moves to another layerwhile pressing a button (e.g., a “Shift” key) and returns to thepreceding layer upon its release. In yet another embodiment, the triggerto switch layers can be defined differently per layer. The user can forexample select the letter “A” in layer 1 to proceed to layer 2, andselect the letter “B” in layer 2 to return to layer 1 or proceed to yetanother layer 3. There can be numerous combinations of layers andtriggers which can be defined to substantially expand the capability ofsingle accessory. Additionally, triggers can be of any kind, tactile,speech, etc.

In another embodiment, method 600 can be adapted so that a user candefine super macros and/or super profiles. A super macro can representnested macros (combinations of macros). Method 600 can be adapted sothat the user can customize the timing for executing nested macros.Similarly, a super profile can represent nested profiles (combinationsof profiles). A super profile can for example comprise sub-profiles,each sub-profile defining associations of actions to input functions ofa particular accessory.

In yet another embodiment, method 600 can be adapted to establish audioprofiles for headset accessories. When a user selects a headsetaccessory such as 314, GUI 301 can be adapted to provide the useroptions to establish a sound output (equalizer) setting to optimizeperformance for a particular gaming application. For instance GUI 101can present an equalizer so that the user can raise the volume of highfrequencies to an enemy's footsteps from a longer distance in a gamingapplication.

The foregoing embodiments are a subset of possible embodimentscontemplated by the present disclosure. Other suitable modifications canbe applied to the present disclosure. Accordingly, the reader isdirected to the claims for a fuller understanding of the breadth andscope of the present disclosure.

FIG. 9 depicts an exemplary diagrammatic representation of a machine inthe form of a computer system 900 within which a set of instructions,when executed, may cause the machine to perform any one or more of themethodologies discussed above. In some embodiments, the machine operatesas a standalone device. In some embodiments, the machine may beconnected (e.g., using a network) to other machines. In a networkeddeployment, the machine may operate in the capacity of a server or aclient user machine in server-client user network environment, or as apeer machine in a peer-to-peer (or distributed) network environment.

The machine may comprise a server computer, a client user computer, apersonal computer (PC), a tablet PC, a laptop computer, a desktopcomputer, a control system, a network router, switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. It will beunderstood that a device of the present disclosure includes broadly anyelectronic device that provides voice, video or data communication.Further, while a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein.

The computer system 900 may include a processor 902 (e.g., a centralprocessing unit (CPU), a graphics processing unit (GPU, or both), a mainmemory 904 and a static memory 906, which communicate with each othervia a bus 908. The computer system 900 may further include a videodisplay unit 910 (e.g., a liquid crystal display (LCD), a flat panel, asolid state display, or a cathode ray tube (CRT)). The computer system900 may include an input device 912 (e.g., a keyboard), a cursor controldevice 914 (e.g., a mouse), a disk drive unit 916, a signal generationdevice 918 (e.g., a speaker or remote control) and a network interfacedevice 920.

The disk drive unit 916 may include a machine-readable medium 922 onwhich is stored one or more sets of instructions (e.g., software 924)embodying any one or more of the methodologies or functions describedherein, including those methods illustrated above. The instructions 924may also reside, completely or at least partially, within the mainmemory 904, the static memory 906, and/or within the processor 902during execution thereof by the computer system 900. The main memory 904and the processor 902 also may constitute machine-readable media.

Dedicated hardware implementations including, but not limited to,application specific integrated circuits, programmable logic arrays andother hardware devices can likewise be constructed to implement themethods described herein. Applications that may include the apparatusand systems of various embodiments broadly include a variety ofelectronic and computer systems. Some embodiments implement functions intwo or more specific interconnected hardware modules or devices withrelated control and data signals communicated between and through themodules, or as portions of an application-specific integrated circuit.Thus, the example system is applicable to software, firmware, andhardware implementations.

In accordance with various embodiments of the present disclosure, themethods described herein are intended for operation as software programsrunning on a computer processor. Furthermore, software implementationscan include, but not limited to, distributed processing orcomponent/object distributed processing, parallel processing, or virtualmachine processing can also be constructed to implement the methodsdescribed herein.

The present disclosure contemplates a machine readable medium containinginstructions 924, or that which receives and executes instructions 924from a propagated signal so that a device connected to a networkenvironment 926 can send or receive voice, video or data, and tocommunicate over the network 926 using the instructions 924. Theinstructions 924 may further be transmitted or received over a network926 via the network interface device 920.

While the machine-readable medium 922 is shown in an example embodimentto be a single medium, the term “machine-readable medium” should betaken to include a single medium or multiple media (e.g., a centralizedor distributed database, and/or associated caches and servers) thatstore the one or more sets of instructions. The term “machine-readablemedium” shall also be taken to include any medium that is capable ofstoring, encoding or carrying a set of instructions for execution by themachine and that cause the machine to perform any one or more of themethodologies of the present disclosure.

The term “machine-readable medium” shall accordingly be taken toinclude, but not be limited to: solid-state memories such as a memorycard or other package that houses one or more read-only (non-volatile)memories, random access memories, or other re-writable (volatile)memories; magneto-optical or optical medium such as a disk or tape; andany equivalents. Accordingly, the disclosure is considered to includeany one or more of a machine-readable medium or a distribution medium,as listed herein and including art-recognized equivalents and successormedia, in which the software implementations herein are stored.

Although the present specification describes components and functionsimplemented in the embodiments with reference to particular standardsand protocols, the disclosure is not limited to such standards andprotocols. Each of the standards for Internet and other packet switchednetwork transmission (e.g., TCP/IP, UDP/IP, HTML, HTTP) representexamples of the state of the art. Such standards are periodicallysuperseded by faster or more efficient equivalents having essentiallythe same functions. Accordingly, replacement standards and protocolshaving the same functions are considered equivalents.

The illustrations of embodiments described herein are intended toprovide a general understanding of the structure of various embodiments,and they are not intended to serve as a complete description of all theelements and features of apparatus and systems that might make use ofthe structures described herein. Many other embodiments will be apparentto those of skill in the art upon reviewing the above description. Otherembodiments may be utilized and derived therefrom, such that structuraland logical substitutions and changes may be made without departing fromthe scope of this disclosure. Figures are also merely representationaland may not be drawn to scale. Certain proportions thereof may beexaggerated, while others may be minimized Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

Such embodiments of the inventive subject matter may be referred toherein, individually and/or collectively, by the term “invention” merelyfor convenience and without intending to voluntarily limit the scope ofthis application to any single invention or inventive concept if morethan one is in fact disclosed. Thus, although specific embodiments havebeen illustrated and described herein, it should be appreciated that anyarrangement calculated to achieve the same purpose may be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the above description.

The Abstract of the Disclosure is submitted with the understanding thatit will not be used to interpret or limit the scope or meaning of theclaims. In addition, in the foregoing Detailed Description, it can beseen that various features are grouped together in a single embodimentfor the purpose of streamlining the disclosure. This method ofdisclosure is not to be interpreted as reflecting an intention that theclaimed embodiments require more features than are expressly recited ineach claim. Rather, as the following claims reflect, inventive subjectmatter lies in less than all features of a single disclosed embodiment.Thus the following claims are hereby incorporated into the DetailedDescription, with each claim standing on its own as a separately claimedsubject matter.

What is claimed is:
 1. A system, comprising: a processing systemincluding a processor; and a memory that stores executable instructionsthat, when executed by the processing system, facilitate performance ofoperations, comprising: receiving a user input received from a firstaccessory operably coupled with a first computing device, wherein theuser input is associated with a program operating on the first computingdevice; determining a context of the user input based on the program,the context being determined based on graphics presented in the program;generating an audio signal based on a context of the program when theuser input was received on the first computing device, wherein the audiosignal is generated using a multi-lingual library that is remote fromthe one of the first computing device and a second computing device; andproviding the audio signal to a second accessory for presentation to anintended recipient.
 2. The system of claim 1, wherein the secondaccessory is operably coupled with the second computing device, andwherein the first computing device and second computing device areremote from each other.
 3. The system of claim 2, wherein the firstcomputing device and the second computing device are in communicationvia a network.
 4. The system of claim 2, wherein the user input isnon-speech, and wherein the audio signal is generated by the firstcomputing device.
 5. The system of claim 1, wherein the operationsfurther comprise: obtaining an identity of the intended recipient of theuser input; and determining a language for the intended recipient basedon the identity.
 6. The system of claim 1, wherein the first and secondaccessories are operably coupled with and in proximity to the firstcomputing device.
 7. The system of claim 1, wherein the user input isspeech, and wherein the signal is generated based on the speech usingspeech recognition performed.
 8. The system of claim 1, wherein thesecond accessory is operably coupled with the first computing device,and wherein the audio signal is presented as synthesized voice content.9. The system of claim 1, wherein the second accessory is operablycoupled with a second computing device.
 10. The system of claim 1,wherein the first computing device and the second computing device areremote from each other.
 11. The system of claim 1, wherein themulti-lingual library is searched for matching words using the context.12. A method, comprising: transmitting, by a processing systemcomprising a processor, an input to a remote server, wherein the inputwas received by a first accessory operably coupled with a firstcomputing device, and wherein the first computing device is programmedto present a program; translating, by the processing system, the inputinto a first translated message of a first language using amulti-lingual library; providing, by the processing system, the firsttranslated message to a second accessory for presentation to a firstintended recipient; determining, by the processing system, a context ofthe input based on the program, the context being determined based ongraphics presented in the program; providing, by the processing system,a second translated message based on the input to a third accessory forpresentation to the second intended recipient; and receiving, by theprocessing system, the input from the first computing device, whereinthe first and second translated messages are associated with theprogram.
 13. The method of claim 12, further comprising: determining afirst language of a first intended recipient of input based on a firstidentity of the intended recipient; and determining a second language ofa second intended recipient of the input based on a second identity ofthe second intended recipient.
 14. The method of claim 12, wherein theproviding of the first and second translated messages to the second andthird accessories is performed in real-time.
 15. The method of claim 12,wherein the first translated message is presented at the secondaccessory as synthesized voice content.
 16. The method of claim 12,wherein the first translated message is presented at the secondaccessory as text content.
 17. A non-transitory, machine-readablestorage medium, comprising instructions, wherein responsive to executingthe instructions, a processor performs operations comprising: receivinga user input from a computing device, wherein the computing devicepresents a graphical user interface, wherein the graphical userinterface receives user actuations of a first accessory operably coupledwith a computing device while accessing a program on the computingdevice; updating a multi-lingual library based on updates to theprogram, wherein the multi-lingual library comprises a plurality ofwords associated with the program; matching the user input to words ofthe plurality of words of the multi-lingual library to generate atranslated message; updating the multi-lingual library based on words orphrases suggested by the user input; determining a language of anintended recipient of the user input based on an identity of theintended recipient; and providing the translated message to a secondaccessory for presentation to the intended recipient.
 18. Thenon-transitory, machine-readable storage medium of claim 17, wherein thecomputing device obtains a user input that was inputted into the firstaccessory, wherein the first accessory provides a user interface foruser interaction with the program.
 19. The non-transitory,machine-readable storage medium of claim 17, wherein the operationsfurther comprise determining a context of the user input based on theprogram, the context being determined based on graphics presented in thegraphical user interface.
 20. The non-transitory, machine-readablestorage medium of claim 17, wherein the providing of the translatedmessage to the second accessory is performed in real time, and whereinthe first and second accessories comprise one of a keyboard, a gamingpad, a mouse, a gaming console controller, a joystick, a microphone, aheadset with a microphone, or any combination thereof.